A Comprehensive Study of Deep Learning and Traditional Machine Learning Models for Twitter Sentiment Analysis

Abstract

This study presents a comprehensive evaluation of eight different algorithms for sentiment analysis on Twitter data, focusing on the airline industry domain. Using the Twitter US Airline Sentiment dataset, we implemented and compared traditional machine learning methods, lexicon-based approaches and state-of-the-art deep learning models. Our experimental results demonstrate that BERT achieves the highest accuracy of 77.1%, followed by SVM at 74.2% and LSTM at 71.4%. The study provides insights into the strengths and limitations of different approaches for social media sentiment classification and offers practical guidance for selecting appropriate algorithms based on computational requirements and accuracy needs. The research contributes to the growing body of literature on social media analytics and provides actionable insights for practitioners implementing sentiment analysis systems in real-world applications.

Introduction

This study explores Twitter sentiment analysis within the airline industry to understand public opinion and customer satisfaction. Given the high volume of tweets, automated sentiment classification is essential. Airline tweets pose unique challenges due to technical language and emotional content. The research compares eight sentiment analysis algorithms from traditional machine learning (SVM, Random Forest, Naive Bayes), lexicon-based (VADER, TextBlob), and deep learning (CNN, LSTM, BERT) categories using a labeled dataset of 14,640 tweets.

Key findings show BERT, a transformer-based model, achieves the highest accuracy (77.1%), outperforming traditional and lexicon-based models. SVM offers a good balance of accuracy (74.2%) and speed, suitable for real-time applications, while lexicon-based models perform poorly due to context and sarcasm detection limitations. Deep learning models like LSTM outperform CNN, highlighting the importance of sequential processing in sentiment analysis.

The study provides practical guidelines for selecting algorithms based on accuracy, computational resources, and application needs. It also discusses common challenges such as sarcasm, mixed sentiments, and neutral tweet ambiguity. Limitations include dataset scope, language coverage, and computational constraints.

Future work suggests hybrid models, real-time analysis, multi-modal data integration, cross-domain evaluation, and explainable AI to improve sentiment analysis in social media.

Conclusion

This comprehensive study evaluated eight different approaches for Twitter sentiment analysis in the airline industry context. BERT achieved the highest accuracy at 77.1%, demonstrating the effectiveness of transformer-based architectures for understanding complex sentiment expressions in social media text[2][3]. However, traditional approaches like SVM (74.2% accuracy) remain competitive when considering computational efficiency and practical deployment constraints[2]. The results provide actionable insights for practitioners: BERT for maximum accuracy requirements, SVM for balanced performance and speed, and traditional methods for resource-constrained environments. The significant performance gap between supervised learning approaches and lexicon-based methods (VADER: 51.2%, TextBlob: 43.7%) emphasizes the importance of domain-specific training for effective sentiment analysis[2]. Future work should focus on developing hybrid approaches that combine the interpretability of traditional methods with the contextual understanding of deep learning models, while addressing the computational challenges of deploying state-of-the-art models in real-time applications[3][5]. The integration of multiple modalities and cross-domain evaluation will further advance the field of social media sentiment analysis.

References

[1] Twitter US Airline Sentiment. Kaggle. 2019. Available: https://www.kaggle.com/datasets/crowdflower/twitter-airline-sentiment [2] R. Shad, et al. \"Natural Language Processing (NLP) for Sentiment Analysis: A Comparative Study of Machine Learning Algorithms.\" IJAIML, vol. 5, no. 1, 2025, pp. 58-69. [3] A. Patel, et al. \"Sentiment Analysis of Customer Feedback and Reviews for Airline Services using Language Representation Model.\" Procedia Computer Science, 2023. [4] L. Deng, et al. \"Analysis of the Effectiveness of CNN-LSTM Models Incorporating Bert and Attention Mechanisms in Sentiment Analysis of Data Reviews.\" Atlantis Press, 2024. [5] \"An Evaluation of the CNN-LSTM Model\'s Efficacy in Sentiment Analysis.\" IRJMS, 2024. [6] osanseviero/twitter-airline-sentiment. Hugging Face Datasets. 2022. [7] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, \"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,\" NAACL-HLT, 2019. [8] Y. Kim, \"Convolutional Neural Networks for Sentence Classification,\" EMNLP, 2014. [9] S. Hochreiter and J. Schmidhuber, \"Long Short-Term Memory,\" Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997. [10] C. J. Hutto and E. Gilbert, \"VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text,\" ICWSM, 2014.

Copyright

Copyright © 2025 Pyna Sai Teja. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.